{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/jermwatt/machine_learning_refined/blob/main/notes/13_Multilayer_perceptrons/13_4_Backprop.ipynb\n",
    "\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 13: Multi-layer Perceptrons "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains interactive content from an early draft of the university textbook <a href=\"https://github.com/neonwatty/machine-learning-refined/tree/main\">\n",
    "Machine Learning Refined (2nd edition) </a>.\n",
    "\n",
    "The final draft significantly expands on this content and is available for <a href=\"https://github.com/neonwatty/machine-learning-refined/tree/main/chapter_pdfs\"> download as a PDF here</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 13.4 The Backpropogation algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *backpropogation algorithm*, often referred to simply as *backpropogation* or *backprop* for short, is the jargon phrase used in machine learning to describe an approach to computing gradients algorithmically via a computer program that is especially effective for models employing multi-layer perceptron units. Backprop is a special case of a more general algorithm generally referred to as the reverse mode of automatic differentiation. The interested reader is encouraged to see Appendix B for more on automatic differentiation.\n",
    "\n",
    "Automatic differentiation allows us to turn over the tedious burden of computing gradients by hand to the tireless laborer that is the modern computer. In other words, the phrase 'backpropogation' is s effective *calculator for computing gradients* that make computing gradients of virtually any cost function an easy chore.  Indeed this is the  algorithm employed by the *automatic differentiator* software `autograd` we recommend using throughout this text.  Like many technical advances throughout history, automatic differentiation was discovered and re-discovered by different researchers in different areas of science and engineering at different times.  This is precisely why this universally applicable concept - automatic differentiation - is referred to as *backpropogation* in the machine learning community, as this was the name given to it by its discoveres in this field."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "86px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 1,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
